Improving Speech Recognizer Performance

This thesis investigates N-best hypotheses reranking techniques for improving speech recognition accuracy. We have focused on improving the accuracy of a speech recognizer used in a dialog system. Our post-processing approach uses a linear regression model to predict the error rate of each hypothesis from hypothesis features, and then outputs the one that has the lowest (recomputed) error rate. We investigated 15 different features sampled from 3 components of a dialog system: a decoder, a parser and a dialog manager. These features are speech recognizer score, acoustic model score, language model score, N-best word rate, N-best homogeneity with speech recognizer score, N-best homogeneity with language model score, N-best homogeneity with acoustic model score, unparsed words, gap number, fragmentation transitions, highest-in-coverage, slot bigram, conditional slot, expected slots and conditional slot bigram. We also used a linear rescaling with clipping technique to normalize feature values to deal with differences in order of magnitude. A searching strategy was used to discover the optimal feature set for reordering; three search algorithms were examined: stepwise regression, greedy search and brute force search. To improve reranking accuracy and reduce computation we examined techniques for selecting utterances likely to benefit from reranking then applying reranking only to utterances so identified. Besides the conventional performance metric, word error rate, we also proposed concept error rate as an alternative metric. An experiment with human subjects revealed that concept error rate is the metric that better conforms to the criteria used by humans when they evaluated hypotheses quality. The reranking model, that performed the best, combined 6 features together to predict error rate. These 6 features are speech recognizer score, language model score, acoustic model score, slot bigram, N-best homogeneity with speech recognizer score and N-best word rate. This optimal set of features was obtained using greedy search. This model can improve the word error rate significantly beyond the speech recognizer baseline. The reranked word error rate is 11.14%, which is a 2.71% relative improvement from the baseline. The reranked concept error rate is 9.68%, which is a 1.22% relative improvement from the baseline. Adding an utterance selection module into a reranking process did not improve the reranking performance beyond the number achieved by reranking every utterance. However, some selection criteria achieved the same overall error rate by reranking just a small number (8.37%) of the utterances. When comparing the performance of the proposed reranking technique to the performance of a human on the same reranking task, the proposed method did as well as a native speaker, suggesting that an automatic reordering process is quite competitive.